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A decade has passed since A Nation At Risk sounded the first 
warning of a crisis in American education. In the interven- 
ing years, valiant efforts to define and stem the crisis have been 
made, especially in the crucial areas of science, mathematics, 
engineering, and technology (SMET), where numerous public and 
private efforts are underway. In 1993 alone, the Federal Govern- 
ment will spend at least $2.2 billion on education programs specif- 
ically targeting SMET fields. Total Federal spending on SMET 
education — a figure that includes related components of numer- 
ous Federal programs that are not targeted specifically toward 
SMET education — may be approximately $24 billion in 1993. 

Despite efforts to improve the quality and equity of SMET 
education at all educational levels, the nation remains at risk of 
losing its competitive edge. Recognizing the need to enhance the 
coordination of Federal SMET programs, in January 1993 the 
Committee on Education and Human Resources (CEHR) of the 
Federal Coordinating Council for Science, Engineering and 
Technology (FCCSET) implemented a five-year strategic plan. 
Among the first of the plan's agenda items was the appointment 
of an Expert Panel charged with two tasks: (1) to conduct a broad 
review of Federal programs in SMET education, and (2) to assess 
Federal program evaluation efforts. This report presents the 
findings of the Panel, along with its recommendations to bring 
about true reform in Federal SMET education programming. 
The recommendations emphasize improved management and 
coordination of programs, more balanced distribution of existing 
funds, and comprehensive program evaluation. Presented first 
are the Panel's principal findings and recommendations, fol- 
lowed by more specific findings and recommendations related to 
each of the Panel's two charges. For a selected bibliography of 
the reference materials consulted during the preparation of this 
report, see the Sourcebook that accompanies the Expert Panel's 
report. 



Executive 
Summary 

"We are involved in two 
wars in this country: a 
civil war between the 
underclass and the 
privileged in our major 
urban centers and a 
global war to compete for 
economic survival. The 
key to resolving both of 
these conflicts is 
education. " 

Karl S. Pister 

Expert Panel Co-Chair 



A Need for Balance 



"Our educational effort 
has not been raised to 
the plateau of the age 
we live in . . . .We must 
measure it not by what it 
would be easy and 
convenient to do, but by 
what it is necessary to 
do in order that the 
nation may survive and 
flourish. " 

W. Lippman. Atlantic 
Monthly, May 1954 



Analysis of Federal programming for science, mathematics, 
engineering, and technology education— the Panels first charge — 
confirms a distressing truth: coordination of Federal programs 
across agencies and governmental levels and with the private 
sector is insufficient. In addition, Federal core programs in SMET 
education lack balance and coherence. Although the Panel 
applauds the ongoing work of the interagency Committee on 
Education and Human Resources for its coordination efforts, the 
Panel members found that its role needs to be strengthened. 

To resolve this problem, the Panel recommends that the 
Committee develop a management plan based on its goals for 
SMET education. An effective plan would coordinate Federal 
efforts across agencies and education levels, and it would empha- 
size program evaluation and dissemination. Active and continu- 
ous dialogue among all governmental players in SMET education 
is vital and must form the basis of a new Federal culture of 
communication and cooperation. 

The Panel's assessment of Federal program evaluation 
efforts — its second charge — reveals similarly sobering findings, 
in brief, the Panel found that current Federal spending on SMET 
education is not guided by assessments of national need, that few 
Federal programs have been thoroughly evaluated to determine 
their effectiveness, and that funding for evaluation and evalua- 
tion personnel is extremely limited. In addition, the Panel learned 
that current evaluation practices are often inadequate for the 
purposes of improving programs, making informed decisions 
about program retention or expansion, and providing for real 
accountability. 

To address these issues, the Panel recommends improved 
national needs assessment. Rigorous evaluation criteria must 
be adopted to ensure that programs meet identified needs. 
Evaluation results should constitute the knowledge base for 
program planning, revision, and replication within and among 
all Federal agencies. 



The Federal 
Investment 
Portfolio 
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In conducting a broad review of Federal programs in SMET 
education, the Panel identified five major needs that cut across 
all levels of education: to increase participation of students from 
underrepresented populations; to increase research on the effec- 
tive use of technology for SMET education; to disseminate exem- 
plary, effective models; to improve instructor preparation — at all 
educational levels — for teaching; and to encourage research on 
teaching and learning to improve instruction and curricula. 

In reviewing issues that relate specifically to prekindergarten 
through grade 12, the Panel identified a need to establish the 
following guidelines: instructional materials and other projects 
targeted to these students should be developed with the benefit of 
all relevant SMET education research: teacher enhancement pro- 
grams should shift their focus away from short-term experiences 
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toward longer term continuous enrichment necessary for 
excellence; and mentoring should become an important tool in 
teacher education through greater emphasis on training teacher 
leaders. In addition, the Panel urged the provision of additional 
SMET education support for low-income children, the establish- 
ment of national goals for technology education, and the funding 
of improvements in technology education curricula. 

To improve SMET education at the undergraduate level, the 
Expert Panel recommends efforts to enhance the teaching prac- 
tices of faculty and the development of incentives to encourage 
colleges and universities to place greater emphasis on learning 
and instruction. The Panel also recommends, that support for 
programs to develop SMET courses for nonscience majors be 
increased, that newly developed courses be required to incorpo- 
rate up-to-date knowledge about the disciplinary subject matter 
content of each course, and that effective ways to improve student 
learning be implemented. 

At the graduate level, the Expert Panel recommends that 
federally sponsored faculty research grants truly support a well- 
rounded educational experience for graduate assistants. The 
Panel also recommends reallocation of funds to provide more 
support for students in professional, applied SMET areas and 
collaboration among Federal agencies to ensure that the availabil- 
ity and distribution of graduate student support is responsive to 
labor force needs and emerging and interdisciplinary fields of 
SMET. 

In support of community colleges and continuing education 
programs, the Panel urges greater collaboration among iwo- and 
four-year colleges in support of nontraditional students and 
students who are underrepresented in SMET disciplines, as well 
as cooperation among high schools, two-year colleges, four-year 
colleges, and industry in the interests of enhancing technical 
education and training programs. The Expert Panel also recognizes 
the benefits of greater public understanding of SMET issues, and it 
recommends development of a coordinated, ongoing strategic 
plan for increasing public understanding. 



"Public understanding of 
science needs to focus 
on more than just 
classrooms. Media and 
museum programs must 
become important parts 
of the community 

venture in learning. " 

Mary Budd Rowe 
Expert Panel Co-Chair 



After examining the Federal program evaluation activities, 
the Expert Panel concluded that evaluations of SMET education 
programs conducted to date are inadequate and, to a great 
extent, the effects and effectiveness of much of the Federal 
investment remain unexamined. To remedy this situation, the 
Panel recommends the implementation of the evaluation compo- 
nent of the Federal Strategic Plan for SMET education and 
continued monitoring by the Committee on Education and 
Human Resources. In addition, the Panel recommends that 
Federal agencies be required to show significant progress in 
planning and implementing evaluations by the end of fiscal year 
1994; that evaluation efforts be prioritized and coordinated 
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Evaluation 
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". . . the United States 

simply does noi have the 

luxury of supporting the 

wrong programs or failing 

to support the right ones. " 
Expert Panel 



across agencies; that the agencies implement standards of evalu- 
ation practice, using as a base those standards currently being 
revised by the Joint Committee on Standards of EducaVional 
Evaluation; and that evaluations within and across programs be 
based on a systems view, that is, a view that considers key factors 
and influences on program operation and on short- and long-term 
outcomes. 

To support the comprehensive, cross-agency evaluation ef- 
fort recommended by the Panel, the Panel suggests development 
of a plan to explore the types of assessment required for a SMET 
curriculum that fosters critical thinking and measures the 
curriculum's effects. It also recommends that funding for 
evaluation be considered a priority budget item by the agencies 
involved. 

Finally, the Expert Panel recommends that each agency estab- 
lish a minimum core set of statistical indicators to be collected 
and synthesized for similar types of programs; that the indicators 
be augmented by objective, systematic evaluation studies; and 
that Federal agencies encourage local programs to use evalua- 
tion information for program decision making. 

The findings of the Expert Panel and the resulting recommen- 
dations are designed to help improve the Federal effort to educate 
Americans in science, mathematics, engineering, and technology. 
These are areas upon which the civic and economic well-being of 
this country depend. This nation cannot afford to continue 
supporting education programs whose outcomes are untested or 
unknown. The Federal Government can and should play a leader- 
ship role in science, mathematics, engineering, and technology 
education. The Panel recommends that Federal SMET education 
programs be understood and managed as a "portfolio of invest- 
ments," with the ultimate goals being improved quality of life for 
all Americans and greater prosperity for this nation. 



9 

ERLC 



« 



The Federal 
Investment in Science, 
Mathematics, Engineering, 
and Technology Education: 

Where Now? 
What Next? 



Report of the 
Expert Panel for the 
Review of Federal 
Education Programs 
in Science, Mathematics, 
Engineering, and 
Technology 

to the 

Federal Coordinating 
Council for Science, 
Engineering and 
Technology 

Committee on Education 
and Human Resources 



june 1993 



Expert Pane! for the Review of Federal Programs 

in Science, Mathematics, Engineering, and Technology 
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Acting Chair 

Federal Coordinating Council 
for Science, Engineering and 
Technology, Committee on 
Education and Human Resources 

Dear Luther: 

We are pleased to transmit to you The Federal Investment in Science, Mathematics, Engineering, and 
Technology Education: Where Now? What Next? report in fulfillment of the charge to the Expert Panel 
for the Re v iew of Federal Education Programs in Science, Mathematics, Engineering, and Technology. 

A decade ago, the report A Nation At Risk warned that the educational foundations of our society were 
being eroded. Today, our report warns that— although some important first steps have been taken— 
Federal agencies currently are not effectively working together to do all they can and must do to 
reverse the "rising tide of mediocrity/* 

The central theme of the report is that Federal agency efforts in science, mathematics, engineering 
and technology education should be viewed and managed as a portfolio of investments in our nation s 
future. This portfolio needs to be skillfully shaped, maintained, and cultivated. This requires the 
development of a new culture of coordination and cooperation among Federal agencies and, most 
important, a commitment to rigorous, continuous program evaluation and improvement. 

It has been our pleasure to have served our nation together with such distinguished colleagues. 



Respectfully, 




Karl S. Pister 
Co-Chair 
Expert Panel 



Mary Budd Rowe 
Co-Chair 
Expert Panel 
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NATIONAL SCIENCE FOUNDATION 
4201 WILSON BOULEVARD 
ARLINGTON, VIRGINIA 22230 



Honorable John H. Gibbons 
Assistant to the President 

for Science and Technology 
Old Executive Office Building 
Washington, D.C. 20500 

Dear Jack: 

It is with pleasure that I submit to you the report , The Federal Investment 
in Science, Mathematics, Engineering, and Technology Education: Where 
Now? What Next? This report was produced by an independent panel of 
experts chartered under the Federal Coordinating Council for Science, 
Engineering and Technology, Committee on Education and Human Resources. 

The panel reviewed several hundred initiatives in science, mathematics, 
engineering , and technology educat ion from 16 agencies ; examined 
interagency policies and plans? and reviewed the status and role of 
program evaluation. The resulting product is this excellent report and 
sourcebook whose purpose is to guide future planning and interagency 
coordination in education. 

The report reflects two central themes. First, although strides have 
been made to coordinate Federal science, mathematics, engineering, and 
technology education programs under the Committee on Education and Human 
Resources, a greater degree of coordination is necessary and desirable. 
Second, only 20 percent of Federal science, mathematics, engineering, 
and technology education programs have undergone rigorous evaluation for 
effectiveness. Program evaluation efforts should be accelerated, 
strengthened, and coordinated among the agencies. 

I, like the members of the panel, share the view that government policies 
and programs can effectively promote national prosperity and the well- 
being of our people. The panel and its co-chairs have demonstrated 
remarkable leadership in completing their work so efficiently and 
competently. This report provides clear and valuable guidance to improve 
the Federal investment in science, mathematics, engineering, and 
technology education. 



Sincerely, 




Luther S. Williams 
Acting Chair 

Committee on Education and 
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EPA Environmental Protection Agency 

FCCSET Federal Coordinating Council for Science, 
Engineering ana Technology 

HHS Health and Human Services 
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SMET Science, Mathematics, Engineering, 
and Technology 

USDA U.S. Department of Agriculture 
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"The nation that dramatically and boldly led 
the world into the age of technology is 
failing to provide its own children with the 
intellectual tools needed for the Twenty-First 
Century. " 

Educating Americans for the 2 1st Century. National Science Board Commission 
on Pre-College Education in Mathematics, Science, and Technology (1983) 



In 1983. A Xation At Risk jarred the national consciousness. 
Released by the Commission on Excellence in Education, the 
landmark report warned the American people that the nation 
was at risk of losing its competitive edge to countries with 
superior education systems. 

Ten years later. America remains at risk. True, many heard 
the alarm sounded in A Nation At Risk, and many efforts have been 
made to turn the tide. Nevertheless, although some progress 
toward strengthening the quality of American education has 
been achieved, the goals set forth in the report continue to elude 
the nation's grasp. As noted at a 1993 conference commemorat- 
ing the report's l()th anniversary, the recommendations have 
been implemented in a piecemeal fashion, missing the point that 
the system's many parts must work in concert if they ar£ to work 
well. Commission member and Nobel Laureate Glenn Seaborg 
cautioned that. "We are at risk of becoming a Third World country, 
going downhill, losing our competitive edge in a high-tech society, 
and of our people being uneducated to the extent I that ] they can't 
operate intelligently in a democratic society/' 

Today, more than ever, technological competence has become 
crucial to maintaining our nation's position as a leader in global 
affairs and to solidifying our position in the emerging global 
economy. This places a special responsibility on American 
educators to ensure that students at all educational levels gain 
exposure to and have opportunities to pursue high-quality 
science, mathematics, engineering, and technology (SMET) edu- 
cation. In fact. SMET education was a focal point in the National 
Education Goals established by the President and the nation's 
governors in 1990. and it continues to be a concern of many 
Federal and state agencies today. 

Although Federal funding of SMET education constitutes 
only a small portion of the total dollars spent on education in this 
country, the Federal Government nevertheless has expended 
billions of dollars to bring about change and reform in SMET 
education. Still, the impact of current Federal efforts in SMET 
education remains unclear. A potpourri of programs has evolved. 
Federal expenditures are being made with too little overall 
planning and with inadequate evaluation. The resulting array 
of Federal programs boasts some excellent components, but its 
weaknesses and possible redundancies diminish its overall 
effectiveness. 

Recognizing the need to better coordinate what now are 
mostly ad hoc Federal efforts to improve SMET education, the 
Committee on Education and Human Resources (CEHR) of the 
Federal Coordinating Council for Science. Engineering and Tech- 
nology (FCCSET) devised a five-year strategic plan in January 
1993. The plan, which is described in Pathways to Excellence: A 
Federal Strategy for Science. Mathematics. Engineering, and Tech- 
nology Education,* is formulated on the [premise that the Federal 
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". . . while we can take 
justifiable pride in what 
our schools and colleges 
have historically 
accomplished and 
contributed to the United 
States and the well-being 
of Us people, the 
educational foundations 
of our society are 
presently being eroded 
by a rising tide of 
mediocrity that threatens 
our very future as a 
nation and a people. " 

A Nation At Risk. National 
Commission on Excellence 
in Education (1983) 
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Where Now? What Next? 



7f is time for a new 
culture of interaction, 
communication, and #r 
coordination to be 
. developed and sustained 
within and among all 
Federal agencies in the % 
area of education " 

< „ Expert Panel 



Government can best hi > the nation achieve and maintain lead- 
ership in science, math ^rnatics, engineering, and technology by 
furthering SMET education reform. A strong Federal contribution 
to SMET education requires planning, funding, program delivery, 
and program evaluation beyond individual agency authority. It is 
time for a new culture of interaction, communication, and coordi- 
nation to be developed and sustained within and among all Federal 
agencies in the area of education. 

Among the first actions taken by the Committee on Education 
and Human Resources under the 1993 Strategic Plan was the 
creation of an Expert Panel. Supported by the National Science 
Foundation and co-chaired by Dr. Karl S. Pister, Chancellor of the 
University of California, Santa Cruz, and Dr. Mary Budd Rowe of 
Stanford University, the 1 5-member Panel undertook the following 
actions for the Committee: 

■ A broad review of Federal programs in science, mathemat- 
ics, engineering, and technology education at all levels 
(prekindergarten through graduate school). 

■ An assessment of Federal program evaluation efforts. 

This report, The Federal Investment in Science, Mathematics, 
Engineering, and Technology Education: Where Now? What Next?, 
presents the Panel's findings and recommendations relating to 
both charges. The report was drafted by the Panel members, who 
were assigned topics according to their areas of expertise. The 
final report was compiled by Dr. Frances Lawrenz, Special Assis- 
tant to the Panel, under the direction of the co-chairs. For more 
information on the mission of the Expert Panel, its membership, 
and FCCSET CEHR, see the appendix. The report was assembled 
with the help and cooperation of the 13 Federal agencies with 
SMET education-related functions. For a selected bibliography of 
the reference materials consulted during the preparation of this 
report, see the Sourcebook that accompanies this report. 

In addition to this introductory section and a brief conclusion, 
this report has three major sections. "A Need For Balance" 
outlines the two principal findings of the Expert Panel — one 
related tc its review of Federal programs and the other to the 
state of program evaluation. It also contains broad recommen- 
dations for managing and protecting the nation's investment in 
SMET education. 
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"The Federal Investment Portfolio" addresses the first charge 
to the Panel, i.e.. to perform a review of Federal SMET education 
programs. The Panel feels strongly that Federal SMET education 
programs must he managed as a "portfolio of investments" in our 
nation's future. Th^ recommendations contained in this section 
are intended to improve the management and coordination of 
programs and the distribution of existing funds. 

Finally. The Role of Evaluation" addresses the second 
charge to the Panel, i.e., to review Federal evaluation activities. 
The recommendations contained in this section are intended to 
outline a Federal framework for evaluation of SMET education 
programs, increase their frequency, and improve their quality. 

The findings and recommendations offered here begin the 
process of Federal program assessment. The Panel's mandate was 
to provide a limited review based on existing data. A more 
comprehensive review will require in-^epth, long-term investigation. 



"We must strive to meet 
the developmental needs 
of each individual in our 
society to derive the 
maximum benefit from 
the potential capabilities 
of all members of our 
society. " 

LEARNING to Meet the 
Science and Technology 
Challenge, President's 
Council of Advisors on 
Science and Technology 
(1992) 
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In 1983, A Nation A: Risk found little coordination in American 
education policy. Ten years later, the Carnegie Commission 
report, Science, Technology, and Government for a Changing 
World, 2 found similar problems at the Federal level. The present 
report confirms these findings; further, it examines current Federal 
programming within the context of national needs. The Expert 
Panel firmly believes that the United States simply does not have 
the luxury of supporting the wrong programs or failing to support 
the right ones. 

In addition to the more specific findings and recommendations 
set out in the sections that follow, the Expert Panel identified two 
principal findings and associated recommendations related to 
each of its two charges. Although these two findings and recom- 
mendations are broader in scope than others in this report, they 
do not summarize or replace the specific findings and recommen- 
dations established by the Panel. 



A Need For 
Balance 



The principal findings address the Panel's specific charges: 
to broadly review the Federal programs in SMET education and 

to assess Federal evaluation 
efforts. 

i. The Investment Portfolio. 

The Federal commitment of 
dollars to SMET education is 
significant. In 1993alone, $2.2 
billion in Federal funds will be 
expended on nearly 300 "core 
programs" constituted solely 
to support SMET education 
(see Figure 1). If the SMET 



Contributing 
Programs 

$22.2 Billion 



Core 
Programs 

$2.2 Billion 



Principal Findings 



Under- Public 
Graduate Pre-K~12 graduate Understanding 

3% 



42% 



35% 



20% 



Figure 1 

Federal Investment in SMET Education 



•' Science. Technology: and Government for a Changing World. The Concluding Report of the 
Carnegie Commission on Science. Technology, and Government Carnegie Commission on 
Science. Technology, and Government (New York. NY: Carnetfie Commssion, 1993). 
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education components of Federal "contributing pro-ams" 
are included, this sum could be as large as $24 A billion. 
Unfortunately, though, the Federal portfolio of core programs 
is unbalanced and lacks coherence, This situation is the 
result of varying agency missions, a decentralized congres- 
sional resource allocation process, and an overall lack of 
coordination and planning. The lack of coherence and 
balance in programs makes it next to impossible to maintain 
fidelity to the overarching national goals for science, math- 
ematics, engineering, and technology education. Further- 
more, far more is spent on SMET education by the combina- 
tion of state and local authorities and private foundations. 
Many of the conclusions in this report also apply to state, local, 
and private efforts. 

ii. Evaluation of the Investment. Current SMET education 
evaluation practices are often inadequate for the purposes of 
improving programs, making informed decisions about pro- 
gram retention or expansion, or providing for real account- 
ability. Funding for evaluation ($8 million) constitutes less 
than one-half of 1 percent of core Federal funding for SMET 
education, and in fact, just 20 percent of the approximately 300 
core Federal SMET programs have been evaluated (see Figure 2). 



Principal 
Recommendations 



Neither Evaluated 
nor Monitored 
48% 




Figure 2 

Evaluation of CORE Programs 



The principal recommendations presented below also 
address the Panel's two-part charge. 

A. Manage the Investment : The work of the Committee on 
Education and Human Resources and its Federal Stra- 
tegic Plan outlined in Pathways to Excellence consti- 
tute a strong beginning— but a stronger management 
plan is crucial. The management plan should designate 
lead agencies for Federal initiatives in particular areas and 
recommend the merger or phasing out, as well as the devel- 
opment, of new programs as appropriate. 
This management plan must treat Federal 
SMET education programs like a portfolio 
of investments by ensuring that a greater 
proportion of agency programs (1) are 
aligned with overall Strategic Plan goals, 
(2) are coordinated across agencies and 
education levels, (3) use effective strate- 
^fcte^jjj^^^y gies for dissemination, (4) include appro- 

I ^| priate evaluations, and (5) promote equity. 

I Active and continuous dialogue within and 

H^Hjj^^/ among agencies (dialogue that includes 

^^^^^ state, local, and private-sector players when 

appropriate) must be based on a renewed 
commitment to effective communication 
and active coordination of effort. 
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Programs 
Monitored 
32% 



Programs 
Evaluated 
20% 
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B. Improve the investment : National needs assessment 
should underlie program initiatives in science, mathe- 
matics, engineering, and technology education. Pro- 
grams should be evaluated rigorously for effectiveness 
in meeting identified needs. Evaluation results should be 
used as a basis for planning and revising programs and 
should be shared with other Federal agencies. The sharing 
of evaluations and evaluation results among agencies pre- 
vents duplication and wasted effort, opens opportunities for 
collaboration across agencies, and helps to build more 
successful programs within agencies. 
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As it reviewed and discussed Federal SMET programs, the 
Expert Panel identified two types of issues: those that 
apply to SMET education as a whole and those that apply to 
specific areas of SMET education. This section of the report 
presents the findings and recommendations related to both types 
of issues. Although the issues applying to SMET education as a 
whole are presented first, those pertaining to specific areas are 
equally important and pressing. Note: Throughout subsequent 
sections of this report, the Panel s findings are represented in italic 
type; recommendations are depicted in bold. 



The Federal 
Investment 
Portfolio 



Issues in SMET 
Education 



7. 7 The Federal Government now spends more than $236 million 
on core programs targeted toward underrepresented groups. 
Approximately 58 percent ($138 million) of this expenditure 
targets racial or ethnic minorities (see Figure 3). Millions of 
additional Federal dollars are spent on underrepresented 
groups through nontargeted programs. 

Despite continued Federal expenditure, the percentage of minor- 
ity students studying SMET subjects (particularly at the graduate 
level) remains abysmally low. In engineering, the physical 
sciences, and mathematics, 
women are also significantly 
underrepresented. The problem 
of poor retention by under- 
represented populations in SMET 
undergraduate and graduate study 
cannot be viewed apart from the 
troubling fact that inadequate prep- 
aration and/or lack of resources 
in SMET often occurs long before 
these students reach college age. 
The programs that demonstrate 
the most success in improvingpar- 
ticipation of underrepresented 
groups are those that provide ear- 
ly support and bridge one educa- 
tion level to the next. However, 
more Federal dollars per student 
are spent at the college level than 
attheprekindergarten through 12th 
grade (pre-K-12) level. 



Findings and 
Recommendations 




Racial and 

Ethnic 
Minorities 
58% 



Women 
22% 



Unknown' 1 
19% 



Other 
Under- 
represented 
Groups" 
7% 



Persons 

with 
Disabilities 
less than 
1% 



Note: Percents reported total more than 100 percent because some programs target 
multiple underrepresented audiences. 

'At the time of publication, the percentage of funding was targeted at multiple groups 
traditionally underrepresented in SMET and therefore could not be accurately 
attributed to any of the groups indicated here. 

"Other underrepresented groups Include geographically underrepresented groups, 
economically disadvantaged groups, and groups with limited English proficiency. 

Figure 3 

Distribution of Core Program Funding Targeting 
Underrepresented Groups 
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Improving Minority Access to 
Research Careers 

In 1991, just 2.5 percent of science and 
engineering Ph.D.s were awarded to 
African Americans; Hispanics received 
oniy 3.3 percent. The Department of 
Health and Human Services' Minority 
Access to Research Careers program 
provides special research training 
opportunities and Incentives to attract 
minority students to and retain them 
in the biomedical and behavioral 
sciences. The program assists minori- 
ty institutions in developing strong 
undergraduate science curricula, in 
stimulating undergraduate students' 
interest in research careers, and in 
increasing the number of students 
pursuing careers in biomedical and 
behavioral science research. The 
program is comprehensive: it address- 
es different educational levels, from 
high school through undergraduate 
and graduate education, and faculty 
enhancement. 



L 



Pre-K-12 
92% 



Undergraduate 
8% 



Invest in all people : Federally funded programs must actively 
and continually seek ways to improve participation and reten- 
tion of underrepresented populations and to make SMET 
accessible to all citizens. Piecemeal programs will not work; we 
need comprehensive strategies that cross the various education 
levels to ensure that the educational system as a whole provides 
continuous support for equity goals. The Federal Government 
should collaborate with state, local, and private organizations to 
expand student support programs in SMET education, particularly 
among underrepresented groups. Funded programs must sustain 
students' confidence and aptitude throughout high school and 
beyond, encouraging movement from one educational milestone to 
the next. 

/. 2 The Federal Government cutrently spends about $25 million on 
educational technology, distributed as shown in Figure 4. 
Federal funds support research and development on applica- 
tions ranging from computer-based instruction to electronic 
information networks to CD-ROMs. Communications technolo- 
gies such as computer networking, cable television networks, 
andsatellite broadcasting have the potential to increase greatly — 
at all educational levels — the availability of and access to SMET 
educational programs and activities. Some of these capabili- 
ties are widely used by industry in training their employees. 
Despite the emergence of promising educational technologies, 
however, not enough is known about the settings in which they 
may be most effective. 

Invest i n effective tools of discovery : The Federal Govern- 
ment should support aggressive research, development, and 
evaluation efforts to determine the types of educational tech- 
nologies that are most effective in different settings and in 
advancing different goals. This research effort is essential to 
ensure the availability of a rich array of practical, effective source 
materials and technological tools and their appropriate uses at all 
educational levels, including adult and continuing education. Links 

between industrial capabilities in 
educational technologies and com- 
munication and SMET education 
needs should also be explored. 
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Graduate 
0% 



Public 
Understanding 
0% 



Figure 4 

Distribution of Core Programs Funding for 
Educational Technologies 
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! Less than I percent of core Federal funding for SMET is allocated 
for dissemination. Valuable educational resources developed 
with Federal funding— including effective educational and instruc- 
tional practices, quality curricula and other educational prod- 
ucts, and information about those resources— have not been 
shared effectively. Furthermore, concise information about the 
availability of the many Federal programs for instructors and 
students is difficult to obtain. This situation is inefficient and 
costly and weakens the entire Federal program portfolio, 

One way to maximize the effect of SMET education resources is 
to improve dissemination. The Federal Government has not 
fully leveraged all of the available resources to support SMET 
goals. Often, Federal SMET education initiatives do not pro- 
mote cooperation among the various segments of our society. 
Few effective alliances among Federal, state, and community 
partners have been formed. 

a. Leverage th e lnye s,tment,^ Federal SMET 
education programs should constructively exploit not only 
available funds but also existing programmatic resources and 
infrastructures when appropriate— personnel, training tools, 
data sources, and management information systems— across 
agencies, other governmental levels, businesses, and com- 
munities. Such leveraging would allow the Federal Government 
to achieve its SMET education goals and reach its desired audi- 
ences more rapidly, efficiently, and cost-effectively, Leadership is 
especially important in promoting alliances among state and local 
agencies and community-based partners in disseminating— through 
multiple means— information about innovative practices that im- 
prove the depth and scope of general SMET literacy. Leadership 
is also important in encouraging a nationally held belief in the value 
and importance of SMET education. 

b. Market the investment: Aggressive, targeted dissemination of 
information relevant to particular audiences— on professional 
and student opportunities, curriculum reforms, research 
results, and implications for practice — should become the 
paramount feature of the Federal Government's commitment 
to demonstrating leadership in SMET education at all levels. 

A Little Federal funding (less than $15 million) has been allocated 
for the initial teaching preparation of instructors at all levels: 
pre-K-12 through graduate education, adult and continuing 
education, and public understanding of SMET Preparation of 
pre-K-12 teachers often fails to include relevant research 
findings on learning and instruction, training in the use of 
educational technologies, or exciting new materials and cur- 
ricula. Graduate students— many of whom serve as teaching 



"Every science agency of 
the government should 
have an explicit 
education charter 
defining its 
responsibilities to 
address precollege issues 
that lie within the 
agency's special 
technical expertise and 
human resource 

requirements. " 

Carnegie Commission on 
Science, Technology, and 
Government (1991) 



Collaborating for the Dissemination 
of New Science Material 
The Smithsonian Institution s National 
Science Resource Center fosters 
interagency collaboration by dissemi- 
nating agency-developed instructional 
materials about teaching science 
through published guidebooks. The 
center is also developing a computer 
network to provide technical assis- 
tance to schools. In addition, the 
center complements efforts of the 
Department of Education and the 
National Science Foundation by 
disseminating resources produced 
with those agencies' support and by 
working in conjunction with their 
dissemination activities. 



"If getting better schools 
means doing better what 
we now do, the job is 
relatively easy. But if it 
means living up to our 
rhetoric about what 
education should be, 
then the challenge is one 
of the greatest this 

nation has faced. " 

J. Good lad. "Toward a 
More Perfect Union," 
State Education wader, 
11)86. Vol. 5. No. 2 



New Ways of Learning Engineering 

Nearly 40 Institutions across the 
country are engaged in a program of 
comprehensive, innovative, and 
systemic change in engineering 
education sponsored by the National 
Science Foundation. The Engineering 
Education Coalitions program is 
reforming the way engineering is 
taught and learned through the 
formation of coalitions of two-year 
colleges, universities, and minority 
institutions. Through joint activities, 
coalition members are seeking to 
improve the quality of U.S. under- 
graduate engineering education and to 
increase the number of engineering 
baccalaureates awarded to women 
and underrepresented minorities. 



assistants or become college-level faculty — receive little, if any, 
formal professional education about effective teaching, litis 
lack of emphasis on instructor preparation locks the nation into 
a continued, costly process of enhancing the skills of instructors 
who received inadequate initial preparation. 

InvestJMhMMture: Funding should be reallocated to provide 
more support for teacher preparation programs at all levels. 
Pre-K-1 2 SMET teacher preparation should include support during 
the critical first years of teaching careers, Programs to educate 
graduate students—as well as current college and university 
faculty members — about new developments in cognitive science 
and effective teaching strategies should be developed. Program 
guidelines should ensure that instructors in all educational areas 
are sensitive to diversity, including multicultural issues and lan- 
guage differences, 

1.5 Research on SMET teaching and learning can form the basis for 
reform in instruction, assessment of student learning, curricu- 
lum development, and other areas. Unfortunately, Federal 
funding is low (about $10 million in core funding) and is too 
concentrated in a limited set of research topics. 

a. Invest in learning what works: Funding for SMET education 
research should be increased, and its focus should be broad- 
ened to include a wider range of research topics and learning 
contexts and the effects of both on SMET education practice. 
Research should be directed toward determining what works for 
diverse groups and in nontraditional settings. Action research — 
making educational reforms through an iterative process of re- 
search and implementation of new approaches — can yield rich 
information about what works and does not work in actual educa- 
tional settings, 

b, Inves t in knowledge sharing: federal programs should en- 
courage the collaboration of SMET education researchers 
with those providing SMET education at all levels, as well as 
with those developing SMET education products. Such a 
collaboration will ensure that current knowledge of SMET teaching 
and learning practice (including subject matter content) is incorpo- 
rated into the planning and implementation of SMET education 
instruction and projects. 
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In addition to the crosscuttinH issues identified in the previous 
section, the Panel addressed a number ot issues specific to par- 
ticular levels or functions of education, beginning with the impor- 
tant early years of learning. Science, mathematics, engineering, 
andtechnologyeclucationatthepre-K-12level is marked by three 
distressing realities: 

■ Many teachers lack adequate understanding of content as 
well as knowledge of effective SMET teaching methods. 

■ Teachers with appropriate skills often lack the resources 
necessary to support effective instruction, 

■ Too many students spend insufficient time on science in 
the early grades— and even less time on technology or 
basic engineering concepts. 

Many students take only the minimum required courses at 
the high school level. Although students tend to spend more time 
learning mathematics than science, engineering, or technology, 
the mathematics proficiency levels reached by many students 
are too low. 

The basic goals of SMET education— which are, first, to 
provide opportunities for all students to develop SMET compe- 
tencies that they can apply in roal-world situations and, second, 
to motivate students from all backgrounds and both genders to 
aspire to careers in these fields— have not been well served by 
traditional programs of SMET instruction at any level. School 
programs should be promoting problem-solving techniques, 
conceptual abilities, and higher level thinking skills to all students, 
they should also be encouraging students to develop positive 
attitudes about SMET. To these ends, learning media, curricula, 
and instructional strategies should encourage students to partici- 
pate actively in SMET-based learning processes and help them 
construct, integrate, and apply SMET knowledge to real-world 
situations. Resources and support systems should facilitate this 
type of instruction, and the students exposed to it should be able 
to develop not only a deeper understanding of concepts and 
processes but also the interest and motivation to pursue further study. 

In addition to encouraging educators to spend more time on 
SMET in school, the Federal investment strategy should encour- 
age greater student exposure to applied sciences, such as envi- 
ronmental or health sciences. Applied sciences integrate topics 
from varied SMET areas, and they help students understand the 
uses of scientific knowledge and methods in their everyday lives. 
Study of the methods and nature of technology is another 
relevant area often neglected in elementary and secondary 
curricula. 



Building Strong 
Foundations: 
SMET Education in 
Prekindergarten 
Through Grade 12 



'Wontraditional 

education is a powerful 

way to motivate students 

and interest parents in 

the serious study of 

mathematics and 

science and to explode 

negative stereotypes of 

science and scientists, " 
Carnegie Commission on 
Science, Technology, and 
Government ( 15)90 
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"A curriculum is more 
for teachers than it is for 
pupils. If it cannot 
t change, move, perturb, 
inform teachers, it will 
have no effect on those 
whom they teach, " 

J.S. Bruner, The Process 
of Education (1977) 



Thirty-five percent ($770 million) of core f ederal funding for 
SMET education in 1993 is being spent on pre-K-12 programs, As 
shown in Figure 5. Federal spending for these programs is divided 
among five principal areas. 

■ Teacher enhancement programs further the skills of 
teachers already in the workforce, while teacher prepara- 
tion programs improve the education received by students 
planning to become SMET teachers. Together, these pro- 
grams receive 51 percent (about $395 million) of the pre-K- 
12 portion of core Federal funding for SMET education, 
with most of these funds ($370 million) going to teacher 
enhancement. Only about $25 million is spent for SMET 
teacher preparation. 

m Organization reform and comprehensive programs receive 
17 percent (about $128 million) of core Federal funding for 
pre-K-12 programs. Organization reform programs are 
intended to stimulate change in many or all components of 
a system of SMET education, such as at the city or state 
level. Comprehensive programs include multiple compo- 
nents designed to address a variety of audiences. 

■ Student support programs receive 16 percent (about $121 
million) of core Federal funding for pre-K-12 programs. 
These programs are designed to motivate students to 
pursue studies in SMET subjects by offering activities 
that demonstrate real-world applications and by provid- 
ing direct financial assistance. 

■ Curriculum improvement programs receive 10 percent 
(about $74 million) of core Federal funding for pre-K-12 

programs. These programs 



Teacher Preparation 
and Enhancement 

51% 



Educational 
Technologies 

3% 



Curriculum 
Improvement 

10% 




Student 
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Other 
3% 

Figure 5 

Distribution of Core Program Funding 
for Pre-K-12 by Functional Area 
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and Comprehensive 
Programs 
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develop new or improved 
materials, courses, and 
curricula and provide fund- 
ing to develop SMET edu- 
cation standards and tech- 
niques that assess student 
progress. 

Educational technologies 
programs receive three 
percent (about $23 million) 
of Federal funding for 
pre-K-12 programs. These 
programs seek ways in 
which communications 
and information technolo- 
gies can enhance SMET 
education. 



More than 80 percent of the funding for programs at the pre-K-12 
level comes from the Department of Education and the National 
Science Foundation. Figure 6 illustrates the amount of funding that 
each agency provides for programs at the pre-K-12 level by 
functional area. 



2.1 A substantial body of research-based knowledge exists on the 
characteristics of effective science and mathematics curricula 
and appropriate teaching strategies. National standards for cur- 
ricula, teaching practices, and assessment techniques that reflect 
this knowledge are available for mathematics for pre-K-12 and 
are being developed for science. 3 Despite the progress being 
made in this area, curriculum development and teacher enhance- 
ment projects do not always draw upon relevant research about 
learning, teaching, curricula, and assessment. Such research is 
rarely stipulated as part of project design. 
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Findings and 
Recommendations 



The Eisenhower Program: A Unique, 
Locally Based Resource 

The Department of Education's 
Eisenhower State Mathematics and 
Science Education Program is the 
largest Federal program in pre-K-12 
SMET education. For many school 
districts, Eisenhower "formula grants" 
(accorded to each di«*rict on the basis 
of its size) are the only support they 
receive for teacher enhancement and 
curriculum reform. Because school 
districts have local control over the 
disposition of these funds, they have 
considerable flexibility in respond- 
ing to the needs of their constituen- 
cies. 



Figure 6 

Core Program Funding for Pre-K-12 by Agency 
and Functional Area 



ERIC 



'See Curriculum and Evaluation Standards for School Mathematics. National Council of 
Teachers of Mathematics Commission on Standards for School Mathematics (Reston, VA: 
National Council of Teachers of Mathematics. 1989) and Professional Standards for Teaching 
Mathematics. National Council of Teachers of Mathematics Commission on Teaching 
Standards for School Mathematics {Reston. VA: National Council of Teachers in 
Mathematics. 1991.) 
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Informal Science Education: 
Strengthening Classroom Learning 

The National Science Foundation's 
Informal Science Education program 
uses nontraditional approaches to 
spark interest in science and technol- 
ogy in the classroom and among the 
public. Projects include media pro- 
grams like "Square One TV." "3-2-1 
Contact," and "Reading Rainbow"; 
permanent and traveling exhibits at 
museums and science centers; and 
after-school and weekend science 
activities for youth. To forge strong 
links between formal (classroom- 
based) and informal science education 
at the pre-K-12 level, curriculum 
experts develop materials to be used 
with educational television programs 
that complement the formal school 
curricula. Informal science educators 
often provide teacher in-service 
training on techniques for inquiry- 
based, hands-on science teaching. 
Also, museum exhibits incorporate 
classroom hands-on activities that 
reinforce school curricula. 



Youth Conservation Corps 

Since 1970. hundreds of thousands of 
students have participated in the 
Department of the Interior's Youth 
Conservation Corps. The program 
was designed to accomplish needed 
conservation work on public lands, 
provide employment for 15- to 18-year- 
olds of all social and economic back- 
grounds, and promote understanding 
and appreciation of the earths 
environment and the nation's conser- 
vation heritage. The scientific and 
environmental knowledge gained by 
young people during the summer 
session builds on science and 
mathematics lessons learned during 
the school year and promotes 
self-discipline and the formation of 
team-building skills. 



a. Program guidelines should require that instructional materi- 
als and other projects developed for pre-K-12 reflect knowl- 
edge of research outcomes, practices, and standards specifi- 
cally related to science and mathematics learning, attitudes, 
motivation, and instructional strategies. Collaboration among 
content area experts and instructional experts from all educa- 
tional levels should also be emphasized. 

b. All federally funded curriculum development and teacher 
enhancement projects should demonstrate — prior to funding— 
a viable plan for incorporating relevant SMET education 
research. 

2.2 Federal teacher enhancement programs are designed to update 
and renew teacher knowledge of content and of appropriate 
curricula and teaching practices. An increasing number of 
programs are showing good multiplier effects by using lead 
teachers to provide education for other teachers. These excel- 
lent teachers expand their knowledge and mentoring skills and 
then, in turn, help their colleagues. 

However, teacher enhancement support is usually provided for 
one-time projects designed to "fix" a perceived problem. It is 
not clear how much follow-up takes place. Nor is it clear 
whether sufficient education is being delivered to groups whose 
knowledgeable participation in upgrading SMET opportunities 
and achievement is important for success (e.g., administrators, 
key board members, etc.). Efforts to bring together all of the 
diverse groups critical to implementing change are rarely 
required under existing programs. 

a. Teacher enhancement programs should be reformulated to 
emphasize the continuous enrichment necessary for excel- 
lence in SMET teaching, rather than being designed to "fix" 
problems. 

b. Program guidelines should require the development and in- 
school support of teacher leaders to provide mentoring and 
education for their colleagues. 

c. SMET programs should include awareness components for 
parents, administrators, school board members, and other 
community members. Better awareness among these groups 
will stimulate understanding of and commitment to SMET 
education and the investments needed to enhance SMET 
offerings. 

2.3 Although some Federal education programs target SMET, 
other important opportunities are missed. Unfortunately, 
many currently funded Federal programs for children (e.g., 
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Head Start ) do not include SMET education. Youngchildren are 
naturally curious and eager to understand the world around 
them; early exposure to age-appropriate, inquiry-based science 
and mathematics curricula provides the foundation on which 
later understanding rests. For older children, this foundation 
can be expanded through direct experience with SMET activi- 
ties. 

Federal programs intended to provide additional support for 
low-income children (e.g., Chapter I and Head Start) should 
include rich early science- and mathematics-related experi- 
ences among the basic criteria required for funding. 

2 A Technology itself is rarely a subject of study in our schools and 
colleges; even in science curricula, issues related to technology 
studies are barely mentioned. No Federal programs exist for the 
express purpose of promoting student learning about the nature 
of technology. Yet knowledge of the processes and methods of 
technology, as well as of the functions of technology in our 
society, could benefit all students and encourage more of them 
to pursue technologically demanding careers. 

a. Funding should be provided to establish national goals for 
technology education and a curricular plan for their imple- 
mentation. Such curricula, if implemented, would increase 
students 1 understanding of technology and encourage more 
students to pursue careers as technicians or engineers. 

b. The Federal Government should fund curriculum improve- 
ment in the area of technology. 



Continuous Enrichment for 
Elementary Through College 

The Mathematics, Engineering, and 
Science Achievement Program, which 
originated in California, engages 
students as early as the third grade. 
Students are continuously encouraged 
and assisted throughout high school 
to prepare themselves for college 
entrance. Although the Mathematics, 
Engineering, and Science Achievement 
Program is not a Federal program per 
se, it is supported in part hy Federal 
funds and provides continuous sup- 
port along the educational continuum 
by building alliances among schools, 
colleges, parents, and businesses. 
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A dramatic drop in SMET participation occurs in the nation's 
institutions of higher education. If this situation is to be improved, 
undergraduate educators in science, mathematics, engineering, 
and technology must overcome serious obstacles. Instructional 
improvements to meet the needs of our prospective teachers, 
lawyers, and business people as well as those planning SMET 
caieers must be made. 

Although educators now know a great deal about effective 
learning and instruction, college instruction has changed only 
cosmetically during the last century. Traditional SMET teaching 
consists of lectures to passive and often disinterested students 
(especially among nonmajors). Sadly, far too many undergradu- 
ate science laboratory courses that :»iiould introduce students to 
inquiry and discovery, to the experimental paradigms of science, 
and to the uses of SMET knowledge fail to do so. In mathematics 
and the physical sciences, in particular, instruction often con- 
sists of doing repetitive textbook problems rather than teaching 
students to identify underlying concepts and procedures and 
helping them to apply their understanding in diverse contexts. 
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Enhancing Our 
Base: 

Undergraduate and 
Additional Paths to 
SMET Education 

Undergraduate SMET 
Education 



. . the Federal 
Government can 
influence faculty 
priorities by giving more 
attention to curriculum 
development, 
instructional innovation, 
and effective teaching. " 

LEARNING to Meet the 
Science and Technology 
Challenge 



Organization 
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Programs 
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Other 
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Twenty percent of 1993 core Federal funds for SMET education 
are being spent at the undergraduate level (about $428 million ). As 
shown in Figure 7, these funds are expended in four major categories. 

■ Student support programs receive 25 percent (about $108 
million) of the undergraduate SMET funding. These pro- 
grams provide research opportunities, financial assistance, 
and mechanisms to help students pursue SMET studies at 
more advanced levels. 

■ Organization reform and comprehensive programs receive 
23 percent ($98 million) of Federal funding at the under- 
graduate level. Three-quarters of this funding is expended 
on comprehensive programs with multiple components 
for a variety of audiences. The remaining one-quarter of 
Federal funding is targeted toward organization reform 
programs that are designed to affect an entire system. 

■ Faculty enhancement programs receive 2 1 percent (about 
$92 million) of undergraduate SMET funding. These pro- 
grams give college and university faculty members the 
opportunity to broaden their skills in research and subject 
matter knowledge. Faculty members can work with re- 
searchers on ongoing projects or attend seminars, insti- 
tutes, and workshops on topics relevant to their disciplin- 
ary expertise. 

■ Curriculum improvement programs receive 13 percent 
(approximately $55 million) of Federal undergraduate SMET 
funding to reform undergraduate courses and curricula in 
SMET subjects. 

Almost two-thirds of Federal 
SMET funding at the undergrad- 
uate level is provided by the 
National Science Foundation and 
the Department of Defense. Figure 
8 illustrates the amount of funding 
that each agency provides for pro- 
grams at the undergraduate level, 
by functional area. 

Curriculum 
Improvement 
13% 



Student 
Support 
25% 



Faculty 
Enhancement 
21% 

Figure 7 

Distribution of Core Program Funding for the 
Undergraduate Level by Functional Area 
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3. 1 Although more than $90 million is currently allocated to faculty 
enhancement programs, the vast majority of this Federal fund- 
ing is targeted toward the improvement of research opportuni- 
ties. Few mechanisms exist to support college faculty members 
seeking to redesign courses, learn about effective teaching 
methodologies developed for specific disciplines, or explore 
research findings on students as learners. Opportunities and 
incentives for undergraduate teachers to learn about teaching, 
learning, and curricula are not widely available. 

a. Efforts to enhance college faculty should be expanded to 
reach many more faculty, and they should be redirected 
toward improving instructional practices and reforming 
curricula. 

b. Federal programs must provide programmatic and monetary 
incentives that encourage colleges and universities to place a 
higher value on learning and instruction. 



Findings and 
Recommendations 



Helping Undergraduate Faculty 
Improve Their Courses 

The Department of Energy sponsors 
several small programs that pair 
undergraduate faculty with practicing 
researchers in Department of Energy 
laboratories to develop and improve 
undergraduate courses and programs 
in energy and environmental manage- 
ment. Faculty members spend a 
summer or an academic year at the 
laboratories to develop materials, 
including classroom and laboratory 
experiments, and to make other 
enhancements to the undergraduate 
courses they teach. 



3.2 Very few Federal programs provide funds for the improvement 
of undergraduate-level courses in SMET areas. Existing SMET 
undergraduate courses, especially at the introductory level, do 



FY 1993 Budget 
(Millions of Dollars) 




Figure 8 

Core Program Funding for the Undergraduate Level 
by Agency and Functional Area 
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Improving Engineering Courses 

Nearly 50 universities are supported 
by the National Aeronautics and 
Space Administrations Advanced 
Design Program which is reforming 
undergraduate study of advanced 
space and aeronautics engineering 
design through specially initiated 
courses and curricula. Originally 
conceived as a way to incorporate 
space and aeronautics topics into 
senior engineering courses, the 
program now also promotes inter- 
disciplinary and cross-disciplinary 
topics and emphasizes the teaching of 
engineering design. 



not reflect or keep pace with advances in teaching and learning 
nor with technologies and tools for problem solving. Further- 
more, although institutions often add new courses at the 
advanced level to reflect new knowledge within a discipline, 
they are slow to recognize new disciplines and interdisciplinary 
areas. For students who do not intend to specialize in SMET 
areas — such as prospective elementary school teachers, law- 
yers, or business managers — courses rarely provide a clear 
appreciation of the methods of science and technology or 
enhance the ability of students to use these methods in their 
daily lives. 

Priority in curriculum improvement support should be given to 
programs that develop SMET courses for nonscience majors, 
especially in emerging disciplines or interdisciplinary fields. 

Federal support guidelines for curriculum development should 
require that newly developed courses incorporate up-to-date 
knowledge about the discipline and effective ways to translate 
that knowledge to students. 



Community Colleges and Since World War IK an extraordinary number of new opportu- 

Adult and Continuing nities for education and training in the sciences, mathematics, 
Education engineering, and technology have become available through 
community college and adult and continuing education pro- 
grams. These programs serve a variety of functions and audi- 
ences, from those seeking basic SMET literacy to highly skilled 
workers seeking advanced training or recertification in their 
fields. The range of programs must continue to meet these varied 
needs. 

Approximately 1,300 regionally accredited community, tech- 
nical, and junior colleges serve more than five million students. 
In 1990, about 44 percent of the nearly 12 million undergraduate 
students in the United States were attending two-year institu- 
tions. The student population of these colleges is diverse racially, 
economically, and in student age. Only about one-third of two- 
year college students are preparing for transfer to four-year 
institutions. 

Certification programs in technical fields, along with retraining 
for technical workers, are important services offered almost exclu- 
sively by two-year institutions. Industry and business partner- 
ships involving shared facilities, donated or consigned equip- 
ment, and cost sharing have helped to keep two-year college 
programs at the forefront of technology training, while high school 
and four-year college and university programs have fallen behind. 
Many community college faculty are also practitioners who can 
offer students regular contact with the professional environments 
of their fields. 
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X^Adult and continuing education programs occur in a variety of 
venues, including employer-funded programs in the workplace, 
programs run by school districts or community colleges, and 
media programs. The Federal role in adult and continuing educa- 
tion has been minimal. The opportunity exists to do more, 
particularly in the area of making existing technical training 
programs more available, perhaps through distance learning 
networks. 
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4.1 Community and junior colleges serve a much more diverse 
clientele than four-year colleges or universities. This diversity 
includes socioeconomic status, geographic distribution, and 
race and ethnicity. Nearly one-half of the students from groups 
underrepresented in SMET careers who are enrolled in institu- 
tions of higher education are enrolled in two-year colleges. Yet 
two-year institutions receive only six percent of core Federal 
funding for undergraduate SMET education. 

Federal funds should be used to encourage greater collabora- 
tion among two- and four-year colleges for improving pro- 
grams that support nontraditional students and students who 
are underrepresented in SMET disciplines. Such collabora- 
tion would help to meet the specific needs of students from 
these groups and increase the number who pursue — and 
progress in— SMET fields beyond the two-year degree. 

4.2 Little Federal funding is provided for technical education or for 
the involvement of students of all ages in appropriate SMET 
industrial educational opportunities. Such opportunities in- 
clude technical apprenticeships, related work experiences, and 
in-school technical training. 

a. More mutually beneficial collaborations among high schools, 
two- and four-year institutions, and industry should be pur- 
sued. Federal funding should be used to encourage stronger 
information sharing and cooperation, especially with respect 
to curriculum development and student assessment method- 
ologies. Such collaboration could provide students in four- 
year institutions with technical training opportunities that 
those institutions often do not have the capacity to offer. 

b. Federal initiatives to strengthen occupational training in tech- 
nical areas should encourage strong curricular linkages to 
relevant science and mathematics concepts. 

4.3 Community and technical colleges fill important niches in our 
educational system as a whole, especially in the areas of worker 
training and continuing education. Industry relies heavily 
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Findings and 
Recommendations 

Workforce Skills Standards and 
Competencies for a 
High-Perfonnance Economy 

The Department of Labor and the 
Department of Education are working 
on a major effort to improve the work 
readiness of our future labor force. 
Building upon significant contributions 
already made by business, labor, 
educators, and states to identify the 
skills and competencies needed by a 
world-class workforce, the two agen- 
cies are now developing a national 
structure to encourage the use of 
voluntary standards that will make 
these skills and competencies integral 
parts of a high-performance economy. 
The school and training program 
curricula resulting from this initiative 
feature new and wide-ranging techno- 
logical applications as well as innovative 
classroom-workplace arrangements 
involving elementary, secondary, and 
postsecondary education levels. 

Building Human Resources in Energy 
Technology 

Several hundred minority students are 
supported each year under the Minority 
Honors Training and Industrial Assis- 
tance program of the Department of 
Energy. The program provides scholar- 
ship support, academic activities with 
college faculty, and other support that 
helps students develop knowledge and 
skills for careers in energy-related 
technical fields. The program fills an 
important niche in the labor force by 
bringing together several audiences: 
talented African American, Hispanic, and 
Native American students; community 
and technical college programs in 
energy technology; and local industry. 



these programs and in many instances, has developed mutu- 
ally beneficial partnerships with community and technical 
colleges. 

The Federal Government should play a more aggressive 
role in the continuing education of the technical workforce 
through such activities as providing models for duplication, 
funding appropriate research, and opening avenues for 
technical assistance. 



of SMET technological concepts if they are to live productive and healthy 



lives in a continually changing, technologically dependent soci- 
ety. The avenues through which public understanding of SMET 
may be improved and maintained are many and varied. Programs 
to serve decisionmakers, community-linked programs, media pro- 
grams, and programs in museums ail serve different public educa- 
tion functions, A diverse network of these and other resources is 
needed to help improve public understanding of SMET. 

Programs that promote public understanding of SMET account 
for just three percent (about $67 million) of core Federal SMET 
education investment. These programs educate audiences of all 
ages about the impact of science and technology on society, about 
basic SMET concepts, and about the underlying principles, meth- 
ods, and processes of SMET research. As is shown in Figure 9, 1993 
Federal funding for public understanding of SMET is distributed in 
four areas. 

■ Public and community-linked programs receive 65 percent 
(about $43.5 million) of Federal funds set aside to promote 

public understanding of SMET. 



Public Understanding 



Americans must understand fundamental scientific and 



Public and 
Community-Linked 
Programs 
65% 




Programs for 
Decisionmakers 
7% 



These programs incorporate sci- 
ence education into activities of 
daily life and promote better un- 
derstanding of the importance of 
science and scientific research. 




Public 
Information 
Campaigns 



Media 
Resources 
15% 



■ Public information campaigns, 
which receive 13 percent (or 
about $8.7 million) of Federal 
funds set aside to promote pub- 
lic understanding of SMET, are 
specialized segments within 
larger public information cam- 
paigns and individual informa- 
tion sources that are specifically 
targeted toward increasing sci- 
entific knowledge and under- 
standing of scientific principles. 



13% 



Figure 9 

Distribution of Core Program Funding for Public 
Understanding of SMET by Functional Area 




■ Media resources receive 15 percent (about $10 million) of 
Federal funding dedicated to public understanding of SMET. 
The funds are used to provide scientific information to 
increase public awareness and knowledge and to help 
change the public perception of science. 

■ The remaining seven percent (about $4.7 million) funds 
programs for government, corporate, and media decision- 
makers. These programs help them understand scien- 
tific concepts, principles, and issues so that they will be 
able to make informed professional decisions. 

Figure 10 shows that the Department of the Interior provides 
two-thirds of the funding for programs designed to improve the 
general public's understanding of SMET, with the Smithsonian 
Institution, the National Science Foundation, and the Department 
of Energy playing smaller but active roles. 

Much remains to be done in increasing public understanding 
of and interest in SMET. Although society relies heavily on 
technology (indeed, most young children take computers for 
granted), positive images of the people who make and use 
technology are sorely lacking in popular culture. 



"Knowledge of the 
humanities. ..must be 
harnessed to science and 
technology if the latter 
are to remain creative 
and humane just as the 
humanities need to be 
informed by science and 
technology if they are to 
remain relevant to the 
human condition. " 

A Nation At Risk 



FY 1993 Budget 
(Millions of Dollars) 
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Figure 10 

Core Program Funding for Public Understanding of SMET 
by Agency and Functional Area * 
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Findings and 
Recommendations 



Science Behind the Art of Nature 

The National Park Service and the Fish 
and Wildlife Service (both of the 
Department of the Interior) use the 
nation's natural resources as the 
subject and as the classroom for their 
programs. At Yellowstone National 
Park, for example, visitors learn about 
the reintroduction of endangered 
species into wildlife and the implica- 
tions of this process for local flora and 
fauna and for the ecosystem as a 
whole. At the Grand Canyon National 
Park in Arizona, exhibits highlight the 
geological principles behind the 
canyon's formation. These efforts 
foster public understanding and 
appreciation of nature and environ- 
mental science so that visitors can 
make informed, responsible decisions 
about environmental issues that will 
affect them and the generations to come. 



5.1 For some Federal agencies, enhancing SMET literacy and 
promoting positive public attitudes toward SMET study and 
work are relatively new endeavors. Certain federally funded 
programs straddle the line between public information and 
public relations efforts. While public relations programs can 
elicit much-needed funds, it is crucial that the public relations 
agenda not overwhelm the SMET education agenda. Overall, 
Federal efforts at public outreach are underfunded and unbal- 
anced in terms of emphasis. Agencies lack knowledge of actual 
public needs, let alone knowledge of the best mechanisms for 
achieving greater awareness of SMET. 

a. A coordinated, ongoing strategic planning process that uti- 
lizes needs assessments and evaluation to increase public 
understanding of SMET must be designed and implemented. 
Overall Federal priorities that strike a balance between differ- 
ent types of programs and support both large- and small-scale 
efforts as appropriate should be established. 

b. The overall level of effort for public understanding functions 
needs to be increased and coordinated, particularly in high- 
impact areas such as media outreach. 

c. Program priorities related to public understanding should 
clearly distinguish between public relations activities and SMET 
literacy efforts. Major efforts by any agency should be reviewed 
by nongovernmental experts in SMET and public literacy. 



5.2 Federal collaboration with local and private-sector initiatives is 
limited, as is the incorporation of existing community-based 
efforts in Federal program planning. Lack of collaboration at 
the planning stage too often leads to duplication of programs 
and competition within communities or among institutions. 
Local expertise and credibility is necessary to ensure effective- 
ness and leveraging of programs. 

Federal programming initiatives should require links and 
coordination with private-sector efforts and with community- 
based programs. 
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The nation s graduate SMET education system has a reputa- 
tion for excellence, and indeed, it produces researchers who are 
among the best in tue world. Often, however, these excellent 
students learn despite, rather than because of, the teaching in 
their courses. Teaching at the graduate level suffers from many of 
the same problems as undergraduate teaching. Nor is the uneven 
quality of teaching the only problem affecting graduate education. 
The levels, depth, diversity, and preparation of the pool of U.S. 
applicants to graduate schools are also cause for concern. The 
reasons for this situation are not fully understood and need further 
investigation. 

Graduate SMET education accounts for 42 percent (about $922 
million) of core Federal funding for SMET education in 1993. 

As can be seen in Figure 1 1. about 65 percent of Federal SMET 
funding at the graduate level is allocated to student support 
programs such as fellowships (financial awards made directly to 
students for academic support) and traineeships (awards made to 
institutions to support students). These student support opportu- 
nities are available to both predoctoral and postdoctoral students. 
More than 80 percent of Federal funding for graduate-level programs 
is provided by the Departments of Health and Human Services and 
Defense (see Figure 12). 

Fellowships and traineeships play different roles; both can be 
used to target funds toward students from underrepresented 
groups and to promote fields perceived as areas of national need. 
However, because of the disjointed Federal resource allocation 
process and the lack of coordination among agencies, fellowships 
and traineeships — as well as other Federal mechanisms of graduate 
student support — suffer from the fragmented Federal approach to 
human resource development in SMET. Additionally, as currently 
administered, both of these funding mecha- 
nisms are slow to respond to labor market 
fluctuations. With increased dialogue 
among agencies, however, fellowships 
and traineeships can be transformed from 
primarily a student support tool to a mech- 
anism that shapes and enhances the 
national human resource pool. 

Inadditiontofundingtraineeshipsand 
fellowships, Federal dollars support gradu- 
ate students through graduate assistant- 
ships for research grants. The amount of 
research grants that supports graduate 
students is difficult to determine because 
most agencies do not identify the portion 
of their research support that goes direct- 
ly to students. The challenge for agencies 



Sharpening Our 
Expertise: 
Graduate SMET 
Education 



Student 
Support 

65% 




Figure 1 1 



Other 
Programs 

35% 



Distribution of Core Program Funding for the 
\ ». Graduate Level by Functional Area 
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funding competitive research grants is to ensure that the grants 
provide adequate education for the graduate students and do not 
encourage students to pursue fields of study in which employment 
prospects are limited. 



Findings and 
Recommendations 



Making Use of Unique Resources 

The Department of Commerce's 
National institute of Standards and 
Technology/National Research 
Council Postdoctoral Research 
Associateships program provides two- 
year fellowship appointments for 
outstanding scientists and engineers 
chosen through a national competi- 
tion. The appointments provide an 
opportunity for the nation's best 
young scientists, mathematicians, and 
engineers to engage In state-of-the-art 
research in association with senior 
research specialists on the Institute's 
staff, utilizing excellent and often 
unique research facilities, 



FY 1993 Budget 
(Millions of Dollars) 
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6.1 The quality of graduate education is affected not only by the 
quality of the research program available to the student but also 
by a number of other factors. In large laboratories, students may 
seldom see their nominal supervisor and may receive little 
mentoring. Students ran be exploited without receiving appro- 
priate credit for their contributions. Currently the totality of the 
graduate educational experience is neglected because of an 
overemphasis on the student's role as research assistant. The 
Federal Government is demonstrating inadequate leadership in 
addressing these system-wide problems. 

Research program guidelines should require that proposals 
explicitly state the role of the grant in providing a well-rounded 
educational experience for the graduate assistants being 
supported. 

6.2 Students seeking a terminal master's degree, a professional 
degree (such as a masters in engineering), a teaching creden- 
tial, or a Ph.D. degree in SMET education are rarely supported 
by the Federal Government through current fellowship and 
traineeship programs. However, people with these skills are 
crucial for ow economic well-being. 

Federal funding should be reallocated to provide more support 
for students in professional, applied SMET areas. Traineeship 

or fellowship opportunities in 
these areas should be augmented 
or developed to supplement exist- 
ing Department of Education loan 
programs 



100 
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Figure 12 

Core Program Funding for the Graduate Level by Agency 
and Functional Area 
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3 Although considerable Federal funding is available for gradu- 
ate study, the match between national labor force needs and the 
distribution of funds across SMET areas has not been systemati- 
cally considered by the Federal Government 

Agencies must collaborate to ensure that the availability and 
distribution of graduate student support is responsive to labor 
force needs and emerging and interdisciplinary fields of SMET 
rather than to idiosyncratic governmental funding patterns. 



Addressing National Needs 

Based on an extensive study of 
expertise-shortage areas in the food 
and agricultural sciences— and at 
the urging o( business and industry 
leaders— the Department of Agricul- 
ture initiated a graduate traineeships 
program in 1984. It is the only 
Federal program targeted specifi- 
cally to the recruitment and training 
of predoctoral students for critical 
food and agricultural scientific 
positions. Former Fellows today are 
making significant teaching and 
research contributions in their areas 
of expertise. 



3.9 



Evaluation is the process of systematically collecting and 
analyzing data to answer questions about the merit and 
worth of programs and their subparts, Evaluation is an essential 
part of sound management, vital to program decision making and 
to strategic planning for the optimal use of available resources. At 
present, however, evaluation of Federal SMET education pro- 
grams is inadequate; consequently, the effects and effectiveness 
of the significant Federal investment are, to a large extent, 
unexamined. For a majority of Federally funded SMET education 
programs, no evaluation information is available at all (see Figure 
13), or no serious inqui/y beyond anecdotal or self-reported data 
has been made. This disturbing lapse must be addressed immedi- 
ately. The Federal Government cannot continue to spend large 
sums of money without knowing if its programs are accomplishing 
their established goals — or if these goals address national needs 
in SMET education. 

At a fundamental level, Federal programs lack a rational 
basis for strategic planning and decision making. For example, 
how much support for mathematics education is needed relative 
to science education? How much should be invested in pre-K-12 
compared to other educational levels? What is a sensible 
strategy to accomplish public literacy goals? The Federal 




Figure 13 

Number of Core Programs Evaluated or Monitored by Agency 



The Role of 
Evaluation 
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"The Federal Government 

cannot continue to spend 

large sum of money J 

without knowing if its 

programs are - J r : 

accomplishing their 

established goals— or if 

these goals address 

national needs in SMET 

education. " 
E*pert Panel 



Government lacks basic needs assessment data that would enable 
it to determine how much to invest and how the investment should 
be distributed across various agencies, fields, programs, strate- 
gies, and educational levels. 

The size and complexity of many Federal SMET programs 
make rigorous evaluation a challenge. Assessing program effects 
on the target audience (e.g., improved pre-K-12 instructional 
practice that promotes access and learning for all students) 
requires both sophisticated methodological skills and substantial 
resources. Even in less complex programs, proper evaluation 
requires time and skill to collect and synthesize appropriate data 
to answer policy questions. Because so many agencies contribute 
to the accomplishment of broad goals, interagency coordination 
of evaluation efforts is critical. 

It is clear that the Federal Government's evaluation practice 
suffers from insufficient resources needed to address these 
challenges adequately. Evaluation must be addressed in an 
interagency context. Problems to be resolved include fiscal 
considerations, mechanisms for cooperation, and educational 
provisions about evaluation. Many agencies lack the staff time, 
expertise, and capacity to design or monitor studies. In many 
cases where evaluations have been performed, agencies fail to 
consider the implications for program planning and improvement. 

The development of a community within the Federal Govern- 
ment that values evaluation will encourage discussion, criticism, 
and promotion of evaluation activities in SMET education. Such 
a self-critical community will result in peer support, capacity 
building, and more successful evaluations. 



Findings and 
Recommendations 



7. / The recently adopted Federal Strategic Plan for SMET educatiott 
contains several vital program evaluation features: 
m Evaluations will be conducted in a continuous, multiyear 
cycle. 

m Agencies will build the appropriate capacity to monitor 
evaluations. 

m Evaluations will be coordinated and synthesized across 
agencies. 

m An expert panel will advise the agencies. 

The evaluation component of the Federal Strategic Plan for 
SMET education must be implemented. The Evaluation Work- 
ing Group of the interagency Committee on Education and 
Human Resources must continue to monitor and more active- 
ly coordinate evaluation of SMETeducation programs through- 
out all ayencies. 
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7. 2 Current efforts to coordinate evaluation activities across agencies 
are progressing too slowly. More attention must be given to 
developing cost-effective evaluations as well as an interagency 
capability to conduct major evaluation initiatives. 

a. All agencies should show evidence of significant progress in 
planning and implementing evaluations by the end of fiscal 
year 1994. 

b. Evaluation efforts must be prioritized and combined across 
agencies to not only make the most efficient use of existing 
funds but also allow examination of the whole Federal port- 
folio in terms of progress, balance, and responsiveness to 
changing needs. It may be possible to develop template or 
prototype evaluation designs that would streamline some of 
the evaluation process. Agencies with expertise in evaluating 
particular types of programs should be designated to take the 
lead in developing common evaluation designs. 

7.3 The quality, extent, and timeliness of evaluation practice vary 
substantially. Although evaluation design obviously depends in 
part on the nature of the programs themselves, and although no 
single set of methodologies or techniques will be appropriate 
for every type of evaluation, agencies and programs must 
nevertheless meet standards of good evaluation practice. 

Federal agencies should implement standards of evaluation 
practice, using as a base those standards currently being 
revised by the Joint Committee on Standards for Educational 
Evaluation, a coalition of 15 professional organizations con- 
cerned with the quality of evaluations. 4 Several concerns are 
especially relevant with regard to the setting of such standards. 

■ Evaluations should be designed to minimize demands on 
project participants. Strategies that require all partici- 
pants or all recipients to respond to extensive data collec- 
tion procedures should be minimized. 

■ Timeliness is essential for evaluation studies whose 
results are expected to inform Government policy mak- 
ers. This fact requires that current governmental clear- 
ance processes be accelerated. 

■ Information on costs and cost comparisons is critical to 
sound evaluation. A cost-benefit perspective should be 
maintained both for programs and for evaluations. 



A Novel Approach to Program 
Improvement 

The Department of Energy, its 
National Laboratory education staff, 
and the National Center for Improving 
Science Education are designing and 
implementing an evaluation of the 
Department's elementary and second- 
ary SMET education programs. This 
is a unique, capacity-building effort 
that is designed to determine what 
practices work best under which 
circumstances and how those prac- 
tices can improve program quality. 
Different from traditional designs, the 
evaluation contains the following 
components: 

■Templates for each program type 
to assess the extent to which best 
practices are being achieved. The 
templates will help program staff 
identify areas for improvement. 

■ Evaluation and assessment tools 
that can be used by other agencies 
and the private sector when imple- 
menting similar programs. 

■ A capacity-building model that can 
identify issues not anticipated 
when the program was originally 
designed. 



1 See Standards for Evaluations of Educational Programs. Projects, and Materials, the Joint 
Committee on Standards for Educational Evaluations (New York.'W:. McGraw-Hill. 1981). 
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Evaluating Complex and 
Large-Scale Approaches to 
Education Reform 

The National Science Foundation has 
initiated a long-term project to 
evaluate the effectiveness of the 
complex and large-scale Statewide 
Systemic Initiatives program. Created 
in 1991, the program is working to 
reform the way K-12 science and 
mathematics is taught and learned in 
25 states across the nation. The 
National Science Foundation is taking 
a multifaceted approach to evaluating 
the program's effectiveness. In addition 
to each state-level evaluation, the 
National Science Foundation will 
perform up to 10 case studies, will 
build a system of statistical indicators 
of program performance, and will 
implement a pilot survey of teachers 
to help determine the classroom-level 
effects of the program. The National 
Science Foundation is also providing 
technical assistance to help states 
implement sound approaches to 
science and mathematics education 
reform and program evaluation. 
Representatives from each state 
initiative meet periodically to share 
their plans for evaluation and to build 
upon what they have collectively 
discovered. 



Assessing Mathematics and 
Science Education 

The Department of Education is 
developing an overall plan to assess 
its mathematics and science 
education programs through 
monitoring and evaluation activities. 
The National Academy of Public 
Administration has assisted in this 
effort by developing suggested 
performance measures for the 
Eisenhower State Mathematics and 
Science Education program, a formula 
grant program designed to improve 
the skills of teachers and the quality 
of instruction in mathematics and 
science education in grades K-12. The 
Department of Education plans to use 
the development of performance 
measures for mathematics and 
science education as a model to guide 
future development of performance 
measures for all program areas. 
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■ Evaluations should be designed with appropriate atten- 
tion to the needs assessment that justifies the program. 

7.4 Because programs and the influences on them are complex, 
evaluations must examine the nature of the programs them- 
selves as well as all intended and unplanned outcomes over 
extended periods of time. This is true for evaluations at all 
levels, from local projects to projects that cut across Federal 
agencies. At present, Federal agencies lack a systematic 
perspective on evaluation that would allow them to revise 
programs on the basis of assumptions, evidence of redun- 
dancies or gaps, or the clarification and validation of 
effective models. 

Evaluations within and across programs should be based on a 
systems view, a view that considers key factors and influences 
on program operation and on short- and long-term outcomes. 
Furthermore, evaluations should encourage the identification 
and dissemination of exemplary practices ano should provide 
those who implement programs with information to help them 
upgrade their programs. 

7.5 Student learning is the ultimate target of a majority of SMET 
programs. Recent studies have shown that the vast majority of 
SMET achievement tests focus on the recall of factual knowl- 
edge rather than on analytical reasoning or problem solving 
ability. Similar problems are evident in student assessment 
methodologies. Indeed, the overall Federal ability to affect 
SMET education has been hampered by inadequate student 
assessment methodologies. 

A comprehensive, cross-agency research plan is needed to 
explore the types of assessment required for a SMET curricu- 
lum that fosters critical thinking and to measure the effects of 
such a curriculum. 

7.6 Time, staff, expertise, and funds are inadequately allocated to 
the evaluation tasks at hand. Good evaluation requires a 
generous yet judicious commitment of resources. 

Funds for evaluation should be priority budget items for 
Federal agencies and for the projects they support. Addition- 
ally, time for learning about how to conduct evaluations and 
for reviewing, synthesizing, and implementing evaluation 
results should be made available to Federal agency staff. 
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7 Many Federal agencies currently collect "indicators " to monitor 
program operations. Indicators are statistics about programs and 
their impacts; as such, they do not substitute for proper evalua- 
tion. However, indicators do play a role in program monitoring. 
They also aid in developing a culture of evaluation that focuses 
on high-priority outcomes and means of attaining them. Unfortu- 
nately, there is no agreed-upon set of indicators across agencies; 
each agency has its own way of collecting statistics. 

a. Evaluation designs across agencies should include a mini- 
mum core set of indicators to be collected and synthesized (in 
conjunction with other information) by program managers for 
similar types of programs. 

b. When indicators are used, they must be augmented by objec- 
tive, systematic evaluation studies. 

c. Federal data collection efforts must not overburden local 
programs but must encourage local programs to use the 
collected information for program decision making. 



Developing a System of Indicators 
to Improve Programs 

The National Aeronautics and Space 
Administration (NASA) is working 
with the National Research Council 
on an 18-month project to develop 
indicators for the evaluation of 
NASA's education programs. The 
objective of the study is to raise the 
quality of data collection, which will 
ultimately help to improve the pro- 
grams. The project is advised by a 
seven-member panel of experts in 
social and behavioral science, 
occupational research, education 
research and practice, and database 
construction. 



As it reviewed the many Federal initiatives in science, mathe- 
matics, engineering, and technology at all educational levels 
in this country, the Expert Panel was struck by the energy and 
commitment of the Federal agencies and countless individuals. 
Across the nation, many positive efforts are making a difference in 
education. In terms of Federal activities, the Federal Coordinating 
Council for Science, Engineering and Technology Committee on 
Education and Human Resources stands as a potentially important 
agent for continued, constructive change in SMET education, and 
numerous Federal agencies are sponsoring positive reforms in SMET 
education. In addition, many talented people— educators, students, 
parents, business people, communityleaders—areactively involved 
in bringing about positive change in SMET education. 

Yet, as the Panel reflected on the decade that has passed since 
the publication of A Nation At Risk, it confirmed that much remains 
to be clone to strengthen and redefine the Federal role in SMET 
education. Too many Americans know virtually nothing of the 
mathematical and scientific concepts and discourse (to say noth- 
ing of the technologies) that enrich contemporary culture, social 
experience, and economic progress. Ill-equipped to participate in 
important policy decisions in fields such as environmental and 
health science; surrounded and often overwhelmed by automated 
information, production and distribution, and communications 
systems; and unnerved by ecological, public health, and other 
social problems for which science and technology seem to lack 
ready solutions, average Americans are losing confidence in the 
value of scientific and technological perspectives. 

It is the strong view of this Panel— after listening to the testi- 
mony of concerned, committed individuals at various Federal 
agencies— that a basic change in the way in which Federal agencies 
view their roles is needed. It is time for a new culture of interaction, 
communication, and coordination to be developed and sustained 
within and among all the agencies in the area of education. This 
new culture will necessitate flexibility in approaches to program 
design, development, implementation, and evaluation. Even well- 
coordinated programs may need to be revised to respond to 
changing external factors, such as new technological develop- 
ments and work force needs. Change will also come in response 
to evaluation results. In the view of the Panel, it is within the 
agencies' power to effect such a culture change— but to do so, the 
agencies will need the fiscal and management support of Congress 
and the White House. 

In summary, the Panel found that todays Federal programs 
in SMET education continue to be burdened by a lack of coordi- 
nation, a lack of evaluation, and a lack of accountability. The 
Federal portfolio of investments in SMET education needs a 
comprehensive, coordinated management plan to provide bal- 
ance and coherence across and within Federal agencies, other 
levels of government, and all areas and levels of SMET education. 



Conclusion 

"The people of the United 
States need to know that 
individuals in our society 
who do not possess the 
levels of skill, literacy, and 
training essential to this 
new era will be effectively 
disenfranchised, not 
simply from the material 
rewards that accompany 
competent performance 
but also from the chance 
to participate fully in our 
national life. " 

A Nation At Risk 



36 



Where Now? What Next? 



"America 's position in 
the world may once have 
been reasonably secure 
with only a few 
exceptionally well- 
trained men and women. 
It is no longer. " 

A Nation At Risk 



Furthermore, programs must be evaluated more rigorously and 
consistently, and the evaluation information must be used more 
effectively. 

It is the hope of the Expert Panel that the recommendations 
presented in this report stimulate future program coordination, 
development, and consolidation; reallocation of funds: and imple- 
mentation of evaluation studies that provide the information 
necessary for making an informed investment in SMET education. 



The findings and recommendations detailed in this report are 
the work of an Kxpert Panel convened by the Federal Coordi- 
nating Council for Science, Engineering and Technology (FCCSET) 
Committee on Education and Human Resources (CEHR). Through 
eight committees, FCCSET coordinates policy for scientific and 
technical issues that affect more than one Federal agency. The 
membership of FCCSET consists of Federal departments and 
agencies with science and/or technology missions and is coordi- 
nated oy the White House Office of Science and Technology Policy. 

In January 1 993, CEHR issued a Strategic Plan based on two years 
of coordinated interagency effort."' The plan addresses the well- 
being of science, mathematics, engineering, and technology (SMET) 
education at all levels and in all sectors. The plan describes the 
Federal Government's five-year planning framework and associated 
milestones that will focus the resources of the 16 participating 
agencies toward improving SMET competence among all students. 
The plan recognizes the importance of program evaluation for 
measuring results, ensuring accountability, and strengthening 
programs. 

The CEHR Strategic Plan also calls for the creation of an Expert 
Panel to provide advice and recommendations on Federal SMET 
education programs and program evaluation. The National 
Science Foundation was given the responsibility to support and 
direct this activity. It is expected that this report of the Panel will 
be used in the Federal Government's five-year planning cycle and 
in shaping the future activities and directions of CEHR in program 
evaluation, program planning, and operation. 

Nominations for Expert Panel membership were accepted from 
the participating agencies of FCCSET CEHR. More than 60 nomina- 
tions were received in five professional categories: scientists, mathe- 
maticians, and engineers: education researchers; education admin- 
istrators: education practitioners: and program evaluators. Of the 
60 nominees, 15 panelists were recommended to CEHR by the 
National Science Foundation. Panelists wereselected based on their 
breadth of experience, past accomplishments, reputations for excel- 
lence, and acquaintance with the Federal science, mathematics, 
engineering, and technology education enterprise. Panelists were 
also selected to ensure balance in gender, race/ethnicity, disciplinary 
background, education level of interest, and geographic distribu- 
tion. Recommendations for Panel membership were approved by 
FCCSET CEHR and its Evaluation Working Group. Drs. Karl Pister and 
Mary Budd Rowe, who were named Panel co-chairpersons, served 
with the 1 3 additional panelists. Biographies of the panelists follow. 



Appendix: 
About the 
Expert Panel 



"We are deeply 
concerned about the 
lost potential 
represented by 
undeveloped talent for 
science, mathematics, 
engineering, and 
technology in America. " 

LEARNING to Meet the 
Science and Technology- 
Challenge 



Ikttltums M Excellenci' A Ecdrral Strato'Uy fnrSvtvna*. Mathematics. Eni>uwvrimi. and 
Ttniumkwx Education, i S Sttt'tue Mathematics. Rnumvennt*. and Teclumh /i>y Education 
Strangle Plan. EY IWH-E)' /.WS Committee on Kducation and Human Resources 
(Washington. D.C.: federal < oordiiwtiiii" Council lor Mieuce. Kn^'mef nun and 1 1- c lmnlin»y 
Committer on Kducation and Human Resources. January i>. I!JM). 
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Biographies 
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Karl Stark Pister, CoChair, Expert Panel; Chancellor, University of 
California, Santa Cruz: and formerly Dean of the College of Engineer- 
ing, University of California, Berkeley, is chairman of the Board on 
Engineering Education for the National Research Council. Dr. Pister 
is a member of the National Academy of Engineering. 

Mary Budd Rowe, Co-Chair, Expert Panel; Professor of Science 
Education, Stanford University, is past president of the National 
Science Teachers Association and formerly a chairperson for the 
American Association for the Advancement of Science (AAAS). She 
is now serving on the Council and the Committee of Council Affairs 
of AAAS. Dr. Rowe produced the first CD ROM available for science 
education, Science Helper K-8. 

Stephen C. Blume, Elementary Science Specialist, St. Tammany 
Parish Public Schools, Slidell, Louisiana, is past president of the 
Society of Elementary Presidential Awardees and author and co- 
author of elementary and middle school science textbooks and 
curricular materials. He was a recipient of the National Presidential 
Award for Excellence in Science Teaching, 1990. 

Patricia Chavez, Statewide Executive Director, New Mexico 
Mathematics, Engineering, Science Achievement (NM MESA, Inc.), is 
responsible for overall administration and advancement of New 
Mexico's successful precollege mathematics, engineering, and sci- 
ence achievement program. She is also the National Vice President 
for the National Association of Precollege Directors, a member of the 
American Association for the Advancement of Science, and a mem- 
ber of the Mathematical Science Education Board. 

Ronald L. Graham, Adjunct Director of Research at AT&T Bell 
Laboratories, is one of the world's leading combinatorial mathe- 
maticians. He is President of the American Mathematical Society 
and Professor of Mathematical Sciences at Rutgers University. 

Joan L. Herman, Associate Director, National Center for Research 
on Evaluation, Standards and Student Testing at UCLA's Graduate 
School of Education, is the author of Tracking Success: A Guide for 
School-Based Evaluation and the editor of Making Schools Work for 
Underachieving Minority Students. 

Ernest Robert House, Professor of Education and Director of the 
Laboratory for Policy Studies at the University of Colorado, Boulder, 
is the author of Professional Evaluation Social Impact and Political 
Consequences. He is the winner of the Lazersfeld Award for Evalua- 
tion Theory in 1990 and the Harold D. Lasweil Prize awarded by 
Policy Sciences in 1989. 

Jacquelyn S. Joyner, Mathematics Instructional Specialist K-12, 
Richmond, Virginia, Public Schools, served as a member of the 
National Advisory Board, Macmillan/McGraw Hill, and was commis- 
sioned by the National Center for Educational Statistics to write a 
paper on the mathematics items of the National Assessment of 
Education Progress (NAEP) examination for 1991. 
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Floretta Dukes McKenzie, President of The McKenzie Group, 
a comprehensive education consulting firm, was formerly supc 
tendent and Chief State School Officer for the District of Columbia 
Public Schools. In the spring of 1990 and 1991, Dr. McKenzie was a 
distinguished visiting professor at Harvard University's Graduate 
School of Education. She is presently Distinguished Urban Educa- 
tor-in-Residence at The American University, Washington, D.C. 

Jose Mestre, Professor of Physics at the University of Massachu- 
setts, Amherst, specializes in cognitive processes pertaining to 
learning science and mathematics and is coauthor of Academic 
Preparation in Science, He has served as chair of the College Boards 
Sciences Advisory Committee and on various national boards, such 
as the National Research Council's Mathematical Sciences Educa- 
tion Board. 

Wendell G. Mohling, Teacher and Outdoor Laboratory Director at 
Shawnee Mission Northwest High School, Kansas, is the NASASpace 
Ambassador from Kansas and a member of the International Faculty 
for the Challenger Center. He was the 1992-93 President of the 
National Science Teachers Association and is a former director of 
the National Science Teachers Association High School Division. 

Michael James Padilla, Chair, Department of Science Education, 
University of Georgia, is an author of numerous articles and books 
on science teacher education. He has been appointed by the 
National Science Teachers Association to various boards of the 
National Council for the Accreditation of Teacher Education and 
currently leads the Georgia Statewide Systemic Initiative. 

Helen R. Quinn. Senior Staff Scientist and Assistant to the Director 
for Education and Public Outreach, Stanford Linear Accelerator 
Center, is a fellow of the American Physical Society (APS), has served 
on the APS Panel on Public Affairs, and is President of the nonprofit 
Contemporary Physics Project. 

Michael Scriven. Consulting Professor, Stanford University 
Graduate School of Education, is a Senior Fellow sponsored by the 
American Educational Research Association and the National Sci- 
ence Foundation. He publishes, teaches, and provides consultation 
in a broad range of disciplines in both the physical and the social 
sciences. He is the editor and author of numerous publications 
including Evaluation Models and the Evaluation Thesaurus. 

James G. Wingate, Vice President for Programs, North Carolina 
Community Colleges, is coauthor of Fundamentals of Probability and 
has been actively involved in the Fund for the Improvement of Post 
Secondary Education, the National Association for Institutional 
Research, and the American Association of Community and Junior 
Colleges. 

Frances Lawrenz. Special Assistant to the Panel Co-Chairs and 
Professor, Director of Graduate Studies in the Department of Cur- 
riculum and Instruction at the University of Minnesota, is the author 
of many articles on science education. She has conducted numerous 
evaluations of science programs. She served twice as a visiting 
scientist for program evaluation at the National Science Foundation. 
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Panel Process 



"All who have meditated 
on the art of governing 
Mankind have been 
convinced that the fate 
of empires depends on 
the education of their 
youth/' 

Aristotle 



The Panel conducted its deliberations through three two-day 
meetings, with work continuing in the intervening time. The 
meetings provided the opportunity to discuss issues and to meet 
with agency representatives to clarify understanding of each 
agency's programs. Panelists focused their deliberations on Fed- 
eral SMET education core programs.' 1 

The Panel used existing materials provided by the agencies 
and FCCSET CEHR. Those materials included— 

■ Relevant plans and strategies provided by each agency and 
FCCSET CEHR. 

■ One-page statements on each program. 

■ A matrix of programs, budgets, and audiences affected. 

■ Written and oral briefings by each agency and FCCSET 
CEHR representatives. 

■ A matrix and supporting narrative on the evaluation 
projects of each agency. 

■ Evaluation reports and program audits. 

■ Other publications, reports, and guides. 

■ Curriculum materials. 

■ Sample surveys. 

m Program guides and inventories. 

■ Information about the condition of SMET education in the 
United States. 

The Panel was divided into three five-member subpanels, 
each with responsibility for reviewing the activities of a subset of 
the agencies in greater detail. The subpaneis were chaired by the 
evaluation specialists on the Panel: Joan Herman, Ernest House, 
and Michael Scriven. Subpanel members were assigned one of five 
topical areas to discuss with their counterparts from the other 
subpanels. The five areas covered were elementary and second- 
ary education, undergraduate education, graduate education, 
public understanding of science, and program evaluation. These 
teams helped to summarize Panel discussions of the topical areas, 
ensuring coverage of all agencies within the topical groups. All 
Panel members were actively involved in the deliberations, and 
the final report represents the consensus of Panel members. 



The panel members also considered Federal support for programs contributing to SMET 
education although not explicitly designated or managed as such. 
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Of the 16 FCCSET CEHRagenoies, 13 participated in Expert Panel Agency 
activity. 7 All FCCSET CEHR agencies with congressionally appropri- Participants 
ated programs for SMET education or programs explicitly managed 
as such (core programs) participated in Panel activity. Each of the 
participating agencies was represented during Panel deliberations 
by its FCCSET CEHR Evaluation Working Group member. Panel 
members wish to gratefully acknowledge the dedication and hard 
workof the Evaluation Working Group participants, which made this 
report possible. 

Department of Agriculture (USDA) 

Gwendolyn Lewis 

Department of Commerce (DOC) 

Carol Ann Meares 

Department of Defense (DOD) 

Robert Lockman 

Department of Education (ED) 
Nancy Loy and Rick Davis 

Department of Energy (DOE) 

Margaret Dwyer and Talitha Powell 

Department of Health and Human Services (HHS) 
Patricia Hoben 

Department of the Interior (DOI) 

William Mehojah and Larry Cardwell 

Department of Labor (DOL) 

John Heinberg 

Department of Transportation (DOT) 

Jocelyn Stevenson 

Environmental Protection Agency (EPA) 
Ronald Slotkin 

National Aeronautics and Space Administration (NASA) 
Malcom Phelps 

National Science Foundation (NSF) 
Kenneth J. Travers, Conrad Katzenmeyer, 
and James S. Dietz 

Smithsonian Institution (SI) 

Ann P. Bay 

Executive Secretary, FCCSET CEHR 
Angela M. Phillips 

Director, Expert Panel Project 

James S. Dietz 
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7 Although members ol FCCSET CEHR. the following agencies have no explicitly appropriated 
(core) programs in SMET education and did not participate In the Panel: the Department of 
Housing and Urban Development (HUD). Department ol Justice (DOJ). and Department of 
Veterans Affairs (VA). 
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